<html>
<head><meta charset="utf-8"><title>categorizing crater runs · t-compiler · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/index.html">t-compiler</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html">categorizing crater runs</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="197972096"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197972096" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197972096">(May 18 2020 at 17:50)</a>:</h4>
<p>So <span class="user-mention" data-user-id="121055">@Pietro Albini</span> was talking to me about what might be useful to "auto-categorize" crater runs and we thought we'd move the conversation here.</p>



<a name="197972143"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197972143" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197972143">(May 18 2020 at 17:50)</a>:</h4>
<p>I'm still a tiny bit unclear on what the expectation here is -- are we going to generate a markdown file? A web page?</p>



<a name="197972169"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197972169" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197972169">(May 18 2020 at 17:50)</a>:</h4>
<p>An interactive tool? :)</p>



<a name="197972195"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197972195" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197972195">(May 18 2020 at 17:51)</a>:</h4>
<p>A web page would give us more room to produce different "views" onto the same data</p>



<a name="197972211"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197972211" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Pietro Albini <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197972211">(May 18 2020 at 17:51)</a>:</h4>
<p>for context, a coursemate of mine is working on a thesis on improving the crater reports</p>



<a name="197972230"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197972230" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197972230">(May 18 2020 at 17:51)</a>:</h4>
<p>e.g., I'd love to have things categorized both by the dependency tree (so that you can see how many errors you get in each crate)</p>



<a name="197972249"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197972249" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197972249">(May 18 2020 at 17:51)</a>:</h4>
<p>but also by errors</p>



<a name="197972282"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197972282" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197972282">(May 18 2020 at 17:51)</a>:</h4>
<p>i.e., "this error message broke crates X, Y, and Z"</p>



<a name="197972296"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197972296" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Pietro Albini <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197972296">(May 18 2020 at 17:51)</a>:</h4>
<p>the current ideas is to improve the current report html</p>



<a name="197972308"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197972308" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197972308">(May 18 2020 at 17:51)</a>:</h4>
<p>I'd probably also like some kind of markdown export that gives me a list of crates where I can edit and leave notes inline</p>



<a name="197972402"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197972402" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197972402">(May 18 2020 at 17:52)</a>:</h4>
<p><span class="user-mention silent" data-user-id="121055">Pietro Albini</span> <a href="#narrow/stream/131828-t-compiler/topic/categorizing.20crater.20runs/near/197972296">said</a>:</p>
<blockquote>
<p>the current ideas is to improve the current report html</p>
</blockquote>
<p>I think that's the right starting point</p>



<a name="197972430"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197972430" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197972430">(May 18 2020 at 17:52)</a>:</h4>
<p>cc <span class="user-mention" data-user-id="116113">@lqd</span> and <span class="user-mention" data-user-id="116118">@Matthew Jasper</span>, at least? I feel like these are two people I remember going through big crater runs recently, though I know many have done so</p>



<a name="197972444"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197972444" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Pietro Albini <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197972444">(May 18 2020 at 17:52)</a>:</h4>
<p><span class="user-mention" data-user-id="116122">@simulacrum</span> as well</p>



<a name="197972477"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197972477" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Pietro Albini <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197972477">(May 18 2020 at 17:52)</a>:</h4>
<p>(I'll try to get the person working on the thesis here btw)</p>



<a name="197972757"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197972757" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197972757">(May 18 2020 at 17:55)</a>:</h4>
<p>I'm wondering if we can make...idk...a hackmd? something to sketch out what a sample report might look like</p>



<a name="197972760"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197972760" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197972760">(May 18 2020 at 17:55)</a>:</h4>
<p>Yeah, so I think there's a few avenues to explore:</p>
<ul>
<li>Improving the raw log access, e.g. keybindings that let you advance and sort into manually chosen buckets -- I find a good portion of the overhead today is just from this if you're not using any tooling</li>
<li>Developing auto-categorization tooling<ul>
<li>Probably we should be running in JSON message mode to ease parsing here? Unsure.</li>
<li>Agreed with Niko that there's two main desired modes "reverse dependency tree" and "by error"</li>
</ul>
</li>
</ul>



<a name="197972816"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197972816" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197972816">(May 18 2020 at 17:55)</a>:</h4>
<p>it's worth pointing out that for at least <em>some</em> crater runs</p>



<a name="197972831"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197972831" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197972831">(May 18 2020 at 17:55)</a>:</h4>
<p>i.e., if we use a deny-by-default lint</p>



<a name="197972843"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197972843" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197972843">(May 18 2020 at 17:55)</a>:</h4>
<p>then we are able to categorize dependencies independently from one another</p>



<a name="197972912"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197972912" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197972912">(May 18 2020 at 17:56)</a>:</h4>
<p>so we don't, I think, want to just <em>assume</em> that if a root dependency is broken, then its children can be ignored (but maybe the user can tell us that?)</p>



<a name="197972994"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197972994" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197972994">(May 18 2020 at 17:56)</a>:</h4>
<p>yeah, I'd imagine that we want to get user input on that</p>



<a name="197973027"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197973027" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197973027">(May 18 2020 at 17:57)</a>:</h4>
<p>in the ideal world your tooling just automagically does that</p>



<a name="197973050"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197973050" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197973050">(May 18 2020 at 17:57)</a>:</h4>
<p>e.g. my tool tries to, because it groups by crate name</p>



<a name="197973093"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197973093" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197973093">(May 18 2020 at 17:57)</a>:</h4>
<p>specifically <em>failed</em> crate name, not the official one we were trying to compile</p>



<a name="197973885"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197973885" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Pietro Albini <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197973885">(May 18 2020 at 18:03)</a>:</h4>
<p><span class="user-mention" data-user-id="303450">@Giacomo Pasini</span> is the one making the thesis <span aria-label="wave" class="emoji emoji-1f44b" role="img" title="wave">:wave:</span></p>



<a name="197973960"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197973960" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Giacomo Pasini <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197973960">(May 18 2020 at 18:03)</a>:</h4>
<p>Hello everyone <span aria-label="wave" class="emoji emoji-1f44b" role="img" title="wave">:wave:</span></p>



<a name="197974031"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197974031" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197974031">(May 18 2020 at 18:04)</a>:</h4>
<p>Hi =)</p>



<a name="197974042"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197974042" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197974042">(May 18 2020 at 18:04)</a>:</h4>
<p>Super excited to hear you would be doing work on this!</p>



<a name="197974073"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197974073" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197974073">(May 18 2020 at 18:04)</a>:</h4>
<p><span class="user-mention silent" data-user-id="116122">simulacrum</span> <a href="#narrow/stream/131828-t-compiler/topic/categorizing.20crater.20runs/near/197973093">said</a>:</p>
<blockquote>
<p>specifically <em>failed</em> crate name, not the official one we were trying to compile</p>
</blockquote>
<p>yes, this would be ideal</p>



<a name="197974306"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197974306" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197974306">(May 18 2020 at 18:06)</a>:</h4>
<p>IIRC there's a way to get cargo to output both json and text, which would probably be good to start with -- currently parsing the logs can be pretty annoying</p>



<a name="197974321"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197974321" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197974321">(May 18 2020 at 18:06)</a>:</h4>
<p>yeah</p>



<a name="197974328"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197974328" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197974328">(May 18 2020 at 18:06)</a>:</h4>
<p>rust-analyzer uses this</p>



<a name="197974344"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197974344" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197974344">(May 18 2020 at 18:06)</a>:</h4>
<p>well, maybe it uses only json</p>



<a name="197974349"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197974349" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197974349">(May 18 2020 at 18:06)</a>:</h4>
<p>but in any case</p>



<a name="197974370"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197974370" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Pietro Albini <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197974370">(May 18 2020 at 18:07)</a>:</h4>
<p>the json includes the rendered text in one of its fields, so we can just use that</p>



<a name="197974373"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197974373" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197974373">(May 18 2020 at 18:07)</a>:</h4>
<p>(You can render the text output faithfully from JSON, or at least you're supposed to be able to)</p>



<a name="197974416"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197974416" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Giacomo Pasini <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197974416">(May 18 2020 at 18:07)</a>:</h4>
<p><span class="user-mention silent" data-user-id="121055">Pietro Albini</span> <a href="#narrow/stream/131828-t-compiler/topic/categorizing.20crater.20runs/near/197974370">said</a>:</p>
<blockquote>
<p>the json includes the rendered text in one of its fields, so we can just use that</p>
</blockquote>
<p>Yes, that's what I'm doing right now</p>



<a name="197974593"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197974593" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Giacomo Pasini <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197974593">(May 18 2020 at 18:08)</a>:</h4>
<p>I tell cargo to produce json output and then I put the rendered message in the logs</p>



<a name="197974850"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197974850" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Giacomo Pasini <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197974850">(May 18 2020 at 18:11)</a>:</h4>
<p><span class="user-mention silent" data-user-id="116122">simulacrum</span> <a href="#narrow/stream/131828-t-compiler/topic/categorizing.20crater.20runs/near/197972760">said</a>:</p>
<blockquote>
<p>Yeah, so I think there's a few avenues to explore:</p>
<ul>
<li>Improving the raw log access, e.g. keybindings that let you advance and sort into manually chosen buckets -- I find a good portion of the overhead today is just from this if you're not using any tooling</li>
<li>Developing auto-categorization tooling<ul>
<li>Probably we should be running in JSON message mode to ease parsing here? Unsure.</li>
<li>Agreed with Niko that there's two main desired modes "reverse dependency tree" and "by error"</li>
</ul>
</li>
</ul>
</blockquote>
<p>also, at the moment I should be able to detect build error codes and dependency errors when testing crates</p>



<a name="197974878"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197974878" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197974878">(May 18 2020 at 18:11)</a>:</h4>
<p>those both sound like amazing improvements already</p>



<a name="197974985"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197974985" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Giacomo Pasini <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197974985">(May 18 2020 at 18:12)</a>:</h4>
<p>Not test errors though, I think that it might be quite difficult, they all seem to be pretty different from one another</p>



<a name="197975018"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197975018" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197975018">(May 18 2020 at 18:12)</a>:</h4>
<p>We probably want either two logs or perhaps interleaved JSON and rendered logs? Ideally I think the processing step would be built-in to crater but wouldn't need to happen during the run, since that means that as we try things out we need to wait a few days each time for crater to run</p>



<a name="197975051"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197975051" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197975051">(May 18 2020 at 18:13)</a>:</h4>
<p>I would leave test errors out of scope for now, I don't think there's much you can do in an automated fashion with those</p>



<a name="197975544"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197975544" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Giacomo Pasini <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197975544">(May 18 2020 at 18:17)</a>:</h4>
<p><span class="user-mention silent" data-user-id="116122">simulacrum</span> <a href="#narrow/stream/131828-t-compiler/topic/categorizing.20crater.20runs/near/197975018">said</a>:</p>
<blockquote>
<p>We probably want either two logs or perhaps interleaved JSON and rendered logs? Ideally I think the processing step would be built-in to crater but wouldn't need to happen during the run, since that means that as we try things out we need to wait a few days each time for crater to run</p>
</blockquote>
<p>What is the purpose of having JSON output for the users?</p>



<a name="197975617"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197975617" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197975617">(May 18 2020 at 18:17)</a>:</h4>
<p>well I'm more so saying for you (or whoever maintains the tooling you build)</p>



<a name="197975675"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197975675" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197975675">(May 18 2020 at 18:18)</a>:</h4>
<p>it is much easier to work on this sort of thing if you can test stuff out on real data in minutes rather than days</p>



<a name="197975700"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197975700" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Pietro Albini <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197975700">(May 18 2020 at 18:18)</a>:</h4>
<p>well I expect all the work to have tests in minicrater</p>



<a name="197975702"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197975702" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197975702">(May 18 2020 at 18:18)</a>:</h4>
<p>and since the tooling really wants JSON, not parsing rendered diagnostics, it's nice to have the JSON on hand</p>



<a name="197975726"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197975726" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Pietro Albini <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197975726">(May 18 2020 at 18:18)</a>:</h4>
<p>and those tests are usually relatively quick</p>



<a name="197975743"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197975743" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197975743">(May 18 2020 at 18:18)</a>:</h4>
<p><span class="user-mention" data-user-id="121055">@Pietro Albini</span> sure I mean that when you notice a bug in some crater run and submit a fix in the ideal world we could also go and "regenerate" the rendered output for a crater run</p>



<a name="197975769"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197975769" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Pietro Albini <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197975769">(May 18 2020 at 18:19)</a>:</h4>
<p>ooh</p>



<a name="197975797"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197975797" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197975797">(May 18 2020 at 18:19)</a>:</h4>
<p>(potentially, we don't even store the ASCII and render it on demand, but that might be expensive, not sure)</p>



<a name="197975839"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197975839" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Pietro Albini <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197975839">(May 18 2020 at 18:19)</a>:</h4>
<p>that sounds difficult given the current architecture</p>



<a name="197975977"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197975977" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197975977">(May 18 2020 at 18:20)</a>:</h4>
<p>I mean rendering the json to ascii can probably even be done in JS, realistically</p>



<a name="197976033"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197976033" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197976033">(May 18 2020 at 18:21)</a>:</h4>
<p>(doing something like the CI rendered links we have in rla)</p>



<a name="197976077"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197976077" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Pietro Albini <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197976077">(May 18 2020 at 18:21)</a>:</h4>
<p>hmm</p>



<a name="197976078"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197976078" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197976078">(May 18 2020 at 18:21)</a>:</h4>
<p>doing ad-hoc queries is much easier on the JSON is basically my point</p>



<a name="197976087"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197976087" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197976087">(May 18 2020 at 18:21)</a>:</h4>
<p>and the tool won't be perfect</p>



<a name="197976106"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197976106" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Giacomo Pasini <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197976106">(May 18 2020 at 18:21)</a>:</h4>
<p><span class="user-mention silent" data-user-id="116122">simulacrum</span> <a href="#narrow/stream/131828-t-compiler/topic/categorizing.20crater.20runs/near/197975743">said</a>:</p>
<blockquote>
<p><span class="user-mention silent" data-user-id="121055">Pietro Albini</span> sure I mean that when you notice a bug in some crater run and submit a fix in the ideal world we could also go and "regenerate" the rendered output for a crater run</p>
</blockquote>
<p>How do you usually fix bug in crater runs? I've never actually used crater so a lot of things are new for me</p>



<a name="197976183"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197976183" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197976183">(May 18 2020 at 18:22)</a>:</h4>
<p>well I mean say -- for example -- we realize it would've been nice to group the results by some substring of the error message</p>



<a name="197976193"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197976193" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197976193">(May 18 2020 at 18:22)</a>:</h4>
<p>if we have the JSON on hand, that's easy</p>



<a name="197976210"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197976210" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197976210">(May 18 2020 at 18:22)</a>:</h4>
<p>if we don't... we basically need to rerun the whole thing</p>



<a name="197977004"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197977004" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197977004">(May 18 2020 at 18:28)</a>:</h4>
<p><span class="user-mention silent" data-user-id="116122">simulacrum</span> <a href="#narrow/stream/131828-t-compiler/topic/categorizing.20crater.20runs/near/197976183">said</a>:</p>
<blockquote>
<p>well I mean say -- for example -- we realize it would've been nice to group the results by some substring of the error message</p>
</blockquote>
<p>yeah this is what I was saying to <span class="user-mention" data-user-id="121055">@Pietro Albini</span>, I forget if we included it here-- that sometimes you'll find 2 or 3 related errors that all kind of mean the same thing</p>



<a name="197977016"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197977016" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197977016">(May 18 2020 at 18:28)</a>:</h4>
<p>and it'd be nice to be able to "group" them</p>



<a name="197977034"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197977034" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197977034">(May 18 2020 at 18:28)</a>:</h4>
<p>being able to regenerate the report would be amazing, but I guess it is hard?</p>



<a name="197977113"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197977113" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Giacomo Pasini <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197977113">(May 18 2020 at 18:29)</a>:</h4>
<p>I don't think it is as long as the results are still in the db</p>



<a name="197977126"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197977126" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Giacomo Pasini <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197977126">(May 18 2020 at 18:29)</a>:</h4>
<p>Not sure about how it is uploaded though</p>



<a name="197977190"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197977190" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Pietro Albini <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197977190">(May 18 2020 at 18:30)</a>:</h4>
<p>hmm, I'm not sure how feasible it is to regenerate the report</p>



<a name="197977276"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197977276" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Pietro Albini <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197977276">(May 18 2020 at 18:30)</a>:</h4>
<p><span class="user-mention" data-user-id="116009">@nikomatsakis</span> also mentioned to me being able to generate a markdown report containing the regressions, so it's easy to annotate it and copy/paste it on github</p>



<a name="197977377"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197977377" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197977377">(May 18 2020 at 18:31)</a>:</h4>
<p>I'm basically saying that we don't actually store the ASCII anywhere  -- if you want to look at it you go to some page with some JS which loads the JSON from S3 and renders that in the browser</p>



<a name="197977412"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197977412" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197977412">(May 18 2020 at 18:31)</a>:</h4>
<p>then regenerating reports becomes a matter of running the tool against an archive</p>



<a name="197977525"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197977525" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197977525">(May 18 2020 at 18:32)</a>:</h4>
<p>the report output is just a couple html files at most is what I'm envisioning</p>



<a name="197977834"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197977834" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Giacomo Pasini <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197977834">(May 18 2020 at 18:34)</a>:</h4>
<p><span class="user-mention silent" data-user-id="121055">Pietro Albini</span> <a href="#narrow/stream/131828-t-compiler/topic/categorizing.20crater.20runs/near/197977276">said</a>:</p>
<blockquote>
<p><span class="user-mention silent" data-user-id="116009">nikomatsakis</span> also mentioned to me being able to generate a markdown report containing the regressions, so it's easy to annotate it and copy/paste it on github</p>
</blockquote>
<p>Something like the regressed crate list? maybe with some kind of grouping by error code or other criteria?</p>



<a name="197979170"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/197979170" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Pietro Albini <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#197979170">(May 18 2020 at 18:45)</a>:</h4>
<p>yeah</p>



<a name="198039798"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/198039798" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Giacomo Pasini <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#198039798">(May 19 2020 at 09:12)</a>:</h4>
<p>So, if I understand correctly it would be good to have:</p>
<ul>
<li>detection of build error codes and dependencies, mostly to group by error and identify root regressions.</li>
<li>Markdown regressions report for easier use on github</li>
<li>Group crates into manually defined groups</li>
</ul>



<a name="198039965"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/198039965" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Giacomo Pasini <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#198039965">(May 19 2020 at 09:14)</a>:</h4>
<p>need to investigate the best way to do the last point</p>



<a name="198076403"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/198076403" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> lqd <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#198076403">(May 19 2020 at 14:45)</a>:</h4>
<p><span class="user-mention" data-user-id="116009">@nikomatsakis</span> yeah I had seen this and the ideas seemed to match quite well the difficulties I've personally experienced in the past. (as you may know) both Mark and I have made scripts/tools to do some analysis on the root regressions parts of the problems (and I wanted to interactively categorize error groups as well but that was longer to do than the time it would have saved me, I think). I would say that executing on the plan above would already be a big help, and if anyone needs more ideas, the usual next step for the crater runs is reproducing and minimizing those regressions. That is also a harder task to automate but I've often had success with <a href="https://github.com/jethrogb/rust-reduce/">https://github.com/jethrogb/rust-reduce/</a> sometimes combined with c-reduce, all in all just removing a lot of unneeded code, even if the result is not the most minimal is already a great help</p>



<a name="198082510"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/198082510" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Giacomo Pasini <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#198082510">(May 19 2020 at 15:23)</a>:</h4>
<p>thank you, I'll take a look at those tools</p>



<a name="198247616"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/198247616" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Pietro Albini <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#198247616">(May 20 2020 at 19:43)</a>:</h4>
<p>also cc <span class="user-mention" data-user-id="120791">@RalfJ</span></p>



<a name="198248708"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/198248708" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#198248708">(May 20 2020 at 19:52)</a>:</h4>
<p>What I've been doing by hand twice now is convert the list of regressions into a hackmd where we can collaboratively tick them off one by one after taking a look. unfortunately for my latest case the hackmd is too big if I put all 400 regressions in there.^^<br>
but still, not having to do that manually would be nice. :)</p>



<a name="198248801"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/198248801" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Pietro Albini <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#198248801">(May 20 2020 at 19:53)</a>:</h4>
<p><span class="user-mention" data-user-id="303450">@Giacomo Pasini</span> ^</p>



<a name="198661949"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/198661949" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Giacomo Pasini <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#198661949">(May 25 2020 at 12:20)</a>:</h4>
<p>This morning I talked with <span class="user-mention" data-user-id="121055">@Pietro Albini</span> and we made sort of a plan for future updates. Given that I do not have unlimited time for my thesis I will gradually implement new features in the following order but I may run out of time and some of them may not be developed soon (I won't have time right after the thesis as I will be working as an intern).</p>
<ul>
<li>Detect build errors and dependencies and show them in the report</li>
<li>Markdown report of regressed crates</li>
<li>Interactive grouping of regressed crates</li>
<li>Nicer way to display crates logs</li>
</ul>



<a name="198662269"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/198662269" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Giacomo Pasini <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#198662269">(May 25 2020 at 12:22)</a>:</h4>
<p>right now I have some questions about the visualization of build errors and dependencies in the report, I'll be back with further details about the last points when I will get there</p>



<a name="198663018"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/198663018" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Giacomo Pasini <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#198663018">(May 25 2020 at 12:29)</a>:</h4>
<p>I took a look at <span class="user-mention" data-user-id="116122">@simulacrum</span> work on crater-generate-report and I think that might be a good starting point: for each root regression there's a list of the dependent crates and a brief desc of the error that have caused the regression in first place</p>



<a name="198663193"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/198663193" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#198663193">(May 25 2020 at 12:31)</a>:</h4>
<p>Yep, I think that's a good place to start. Feel free to steal the code (should be MIT/APACHE already but if not should likely be not too hard to adjust as such). Happy to answer questions as well.</p>
<p>It would definitely benefit from JSON and I think I have some local changes I should push up as well.</p>



<a name="198663884"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/198663884" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Giacomo Pasini <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#198663884">(May 25 2020 at 12:38)</a>:</h4>
<p>Should I group root regressions with the same error code? <br>
What about root regressions that fails with more than 1 error code? Let's suppose there's a crate "crate1" that has build errors A and B. Do you prefer to have the crate "replicated" in both categories like:<br>
A (little desc):</p>
<ul>
<li>crate1</li>
<li>other crates that failed due to error A</li>
</ul>
<p>B (little desc):</p>
<ul>
<li>crate1</li>
<li>other crates that failed due to error B</li>
</ul>
<p>or to have more specific groups like:<br>
A (little desc),  B (little desc):</p>
<ul>
<li>crate1</li>
<li>other crates that have the same set of error codes</li>
</ul>



<a name="198665239"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/198665239" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#198665239">(May 25 2020 at 12:52)</a>:</h4>
<p>I think replicating makes sense, at least to start</p>



<a name="198665295"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/198665295" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#198665295">(May 25 2020 at 12:53)</a>:</h4>
<p>though generally there's sort of two modes I guess -- one where you care less about <em>what</em> failed but more so want a general sense of how widely spread the breakage is</p>



<a name="198665327"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/198665327" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#198665327">(May 25 2020 at 12:53)</a>:</h4>
<p>and that's useful for a general assessment of viability for some change</p>



<a name="198665345"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/198665345" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#198665345">(May 25 2020 at 12:53)</a>:</h4>
<p>and then secondarily, you care about "what actually broke" where splitting makes sense</p>



<a name="198667590"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/198667590" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Giacomo Pasini <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#198667590">(May 25 2020 at 13:16)</a>:</h4>
<p><span class="user-mention silent" data-user-id="116122">simulacrum</span> <a href="#narrow/stream/131828-t-compiler/topic/categorizing.20crater.20runs/near/198665295">said</a>:</p>
<blockquote>
<p>though generally there's sort of two modes I guess -- one where you care less about <em>what</em> failed but more so want a general sense of how widely spread the breakage is</p>
</blockquote>
<p>so maybe in this case it would be useful for example just to list root regressed crates and dependent ones?</p>



<a name="198669604"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/198669604" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#198669604">(May 25 2020 at 13:36)</a>:</h4>
<p>hm yeah maybe, not sure</p>



<a name="198798857"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/198798857" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#198798857">(May 26 2020 at 18:02)</a>:</h4>
<p>side note -- one thing that came up just now was that it would be useful to know</p>



<a name="198798870"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/198798870" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#198798870">(May 26 2020 at 18:03)</a>:</h4>
<p>when regressions are occurring in older versions of the crates</p>



<a name="198798920"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/198798920" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#198798920">(May 26 2020 at 18:03)</a>:</h4>
<p>i.e., if this is a regression in 0.1.0 but there is a 1.0.0 available that works...that affects the decision</p>



<a name="198798938"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/198798938" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Pietro Albini <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#198798938">(May 26 2020 at 18:03)</a>:</h4>
<p><span class="user-mention" data-user-id="303450">@Giacomo Pasini</span> ^</p>



<a name="198802072"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/198802072" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Giacomo Pasini <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#198802072">(May 26 2020 at 18:29)</a>:</h4>
<p>hmm, maybe I can check at the time of the report creation if a new version has been released</p>



<a name="198807256"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/198807256" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> lqd <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#198807256">(May 26 2020 at 19:07)</a>:</h4>
<p>I've done this in the past: call the <a href="https://crates.io/api/v1/crates/$identifier">https://crates.io/api/v1/crates/$identifier</a> api to have information about whether the regressed version is the latest (or if there could be semver-compatible releases for example), how old it is (and whether it is active or inactive), and so on</p>



<a name="198807561"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/198807561" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Giacomo Pasini <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#198807561">(May 26 2020 at 19:09)</a>:</h4>
<p>oh perfect, thank you</p>



<a name="198809097"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/198809097" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Pietro Albini <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#198809097">(May 26 2020 at 19:22)</a>:</h4>
<p>btw, let's query the index instead</p>



<a name="198809109"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/198809109" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Pietro Albini <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#198809109">(May 26 2020 at 19:22)</a>:</h4>
<p>otherwise we risk hammering <a href="http://crates.io">crates.io</a></p>



<a name="200141754"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/200141754" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Giacomo Pasini <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#200141754">(Jun 08 2020 at 19:00)</a>:</h4>
<p>I've noticed that crater-generate-report report displays the number of regressed crates from github and from <a href="http://crates.io">crates.io</a> for each root crate. Is it something useful or is a cumulative counter just fine?</p>



<a name="200141850"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/200141850" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Giacomo Pasini <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#200141850">(Jun 08 2020 at 19:01)</a>:</h4>
<p>Also, I don't think I will have time to work on "interactive grouping of regressed crates"</p>



<a name="200142039"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/200142039" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Giacomo Pasini <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#200142039">(Jun 08 2020 at 19:02)</a>:</h4>
<p>The other features should be coming in the next week or two</p>



<a name="200236551"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/200236551" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#200236551">(Jun 09 2020 at 14:31)</a>:</h4>
<p>One thing that <span class="user-mention" data-user-id="116266">@Santiago Pastorino</span> and I were just syaing is that it'd be great if we could make it more convenient to get the names of maintainers to ping</p>



<a name="200606549"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/200606549" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Giacomo Pasini <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#200606549">(Jun 11 2020 at 21:10)</a>:</h4>
<p><span class="user-mention" data-user-id="116009">@nikomatsakis</span>  how do you prefer to structure the markdown report?<br>
I have two options in mind but feel free to suggest other ideas<br>
1) similar to the html report, a "tree" section in which dependencies are highlighted, and a "results" section, in which root crates are categorized based on the error type (with replication if it has more than 1 error)<br>
2) each crate is reported on its own with the full error (no replication) and its dependent crates. If we want to list dependent crates as a sublist similar to crater-generate-report we might have to duplicate a crate if it depends on more than one root regressed crate</p>



<a name="200786448"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/200786448" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Giacomo Pasini <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#200786448">(Jun 13 2020 at 21:28)</a>:</h4>
<p><a href="/user_uploads/4715/9dJKMs8y4D_u8yohzPW_97Fa/image.png">this is the current draft for markdown report (idea n.2)</a></p>
<div class="message_inline_image"><a href="/user_uploads/4715/9dJKMs8y4D_u8yohzPW_97Fa/image.png" title="this is the current draft for markdown report (idea n.2)"><img src="/user_uploads/4715/9dJKMs8y4D_u8yohzPW_97Fa/image.png"></a></div>



<a name="201139793"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/201139793" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#201139793">(Jun 17 2020 at 13:24)</a>:</h4>
<p><span class="user-mention" data-user-id="303450">@Giacomo Pasini</span> that looks nice!</p>



<a name="202969264"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/202969264" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Giacomo Pasini <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#202969264">(Jul 06 2020 at 09:33)</a>:</h4>
<p>The changes should now be deployed, you can see <a href="https://crater-reports.s3.amazonaws.com/pr-73566/index.html">here</a> and <a href="https://crater-reports.s3.amazonaws.com/pr-73461/index.html">here</a> two examples of the current status.</p>



<a name="202969347"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/202969347" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Giacomo Pasini <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#202969347">(Jul 06 2020 at 09:34)</a>:</h4>
<p>there are still some things to fix, like the massive list of crates in the markdown report (currently working on this)</p>



<a name="202970059"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/202970059" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Giacomo Pasini <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#202970059">(Jul 06 2020 at 09:44)</a>:</h4>
<p>Some thoughts on future improvements:</p>
<ul>
<li>As you can see <a href="https://crater-reports.s3.amazonaws.com/pr-73461/index.html">here</a> if the errors don't have an error code they all go into the <code>build failed (unknown)</code> bucket, and that is not ideal</li>
<li>In the same pr, at least 10 versions of ncurses caused regressions in the dependent crates. Might be useful to additionally group all the versions of the same crate under another tab</li>
</ul>



<a name="202970401"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/202970401" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Giacomo Pasini <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#202970401">(Jul 06 2020 at 09:49)</a>:</h4>
<p><span class="user-mention silent" data-user-id="116009">nikomatsakis</span> <a href="#narrow/stream/131828-t-compiler/topic/categorizing.20crater.20runs/near/200236551">said</a>:</p>
<blockquote>
<p>One thing that <span class="user-mention silent" data-user-id="116266">Santiago Pastorino</span> and I were just syaing is that it'd be great if we could make it more convenient to get the names of maintainers to ping</p>
</blockquote>
<p>we decided not to automatically get the list of maintainers though, as it may impact <a href="http://crates.io">crates.io</a> availability</p>



<a name="203054964"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/203054964" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#203054964">(Jul 07 2020 at 00:00)</a>:</h4>
<p><span class="user-mention" data-user-id="303450">@Giacomo Pasini</span> that's very exciting!</p>



<a name="203638151"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/203638151" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#203638151">(Jul 12 2020 at 10:42)</a>:</h4>
<p>this looks amazing! it's not possible to apply this analysis to a past existing run, is it?</p>



<a name="203808691"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/203808691" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Giacomo Pasini <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#203808691">(Jul 14 2020 at 08:14)</a>:</h4>
<p>It depends on how old the experiment is I believe, if it's recent enough maybe <code>@retry-report</code> will do the trick.</p>



<a name="203808733"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/203808733" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Giacomo Pasini <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#203808733">(Jul 14 2020 at 08:15)</a>:</h4>
<p>If it's too old we didn't collect enough information in the first place to categorize the results</p>



<a name="203811531"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/203811531" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Giacomo Pasini <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#203811531">(Jul 14 2020 at 08:53)</a>:</h4>
<p>I don't remember when it was deployed though, maybe <span class="user-mention" data-user-id="121055">@Pietro Albini</span> can help</p>



<a name="203816502"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/203816502" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Pietro Albini <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#203816502">(Jul 14 2020 at 10:05)</a>:</h4>
<p><span class="user-mention" data-user-id="120791">@RalfJ</span> how old is that run?</p>



<a name="203821793"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/203821793" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#203821793">(Jul 14 2020 at 11:22)</a>:</h4>
<p><a href="https://github.com/rust-lang/rust/pull/71274#issuecomment-620227747">https://github.com/rust-lang/rust/pull/71274#issuecomment-620227747</a></p>



<a name="203952379"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/203952379" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Pietro Albini <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#203952379">(Jul 15 2020 at 13:06)</a>:</h4>
<p><span class="user-mention" data-user-id="120791">@RalfJ</span> that's too old :(</p>



<a name="203952671"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/203952671" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Pietro Albini <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#203952671">(Jul 15 2020 at 13:08)</a>:</h4>
<p><span class="user-mention" data-user-id="120791">@RalfJ</span> I'd queue the run again when the queue is shorter</p>



<a name="203953543"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/203953543" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> RalfJ <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#203953543">(Jul 15 2020 at 13:17)</a>:</h4>
<p>It's okay, that would probably need lots of manual checking anyway. So I'll just continue that pass when I find the time.</p>



<a name="204070383"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/204070383" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Pietro Albini <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#204070383">(Jul 16 2020 at 11:14)</a>:</h4>
<p>huge congrats to <span class="user-mention" data-user-id="303450">@Giacomo Pasini</span> for graduating with the maximum grade!</p>



<a name="204070388"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/204070388" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Pietro Albini <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#204070388">(Jul 16 2020 at 11:14)</a>:</h4>
<p>thanks again for working on this as part of your thesis <span aria-label="heart" class="emoji emoji-2764" role="img" title="heart">:heart:</span></p>



<a name="204071226"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/categorizing%20crater%20runs/near/204071226" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Giacomo Pasini <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/categorizing.20crater.20runs.html#204071226">(Jul 16 2020 at 11:26)</a>:</h4>
<p>I would like to thank everyone who helped me in the project, especially <span class="user-mention" data-user-id="121055">@Pietro Albini</span> for introducing me to the Rust community in first place</p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>